Automatic guide apparatus for public transport

ABSTRACT

There is provided an automatic guide apparatus for traffic facilities capable of automatically recognizing the circumstances around the traffic facilities for motor vehicles or the like which change momentarily to automatically guide the operating route of the road or the like. In the automatic guide apparatus for traffic facilities  1,  image data, voice data or the like in connection with the circumstances around the traffic facilities to be operated obtained by the input device  2  are compared with image data, voice data or the like stored in advance in a database by the comparison device  4 . Where the results obtained by comparing the image data, voice data or the like are coincided, the contents of the data are recognized by the recognition device  5,  and the recognized results are informed to an operator or the like in a letter or character, an image, a voice or the like by the output device  6.

TECHNICAL FIELD

The present invention relates to an automatic guide apparatus fortraffic facilities for automatically recognizing the circumstances withrespect to traffic facilities to be operated, informing an operator orthe like of the circumstances in a letter or a character, an image, avoice or the like, guiding an operating route, and directing adequatebehavior in the operation.

BACKGROUND ART

Heretofore, there has not been present an apparatus for automaticallyreading out marks, signs or the like provided on the road, recognizingthe meaning or contents thereof, recognizing the circumstances ofvehicles advancing side by side, passing vehicles, opposing vehicles andthe like which travel on the roads, automatically informing a driver oran occupant of the circumstances with respect to the vehicles, andguiding the road. Accordingly, an occupant sat on the assistant's seatreads marks, signs or the like provided on the road, informs the driverof the contents thereof, or looks at vehicles advancing side by side,passing vehicles, opposing vehicles and the like which travel on theroads to inform the driver of the circumstances with respect to the ownvehicle, or as the case may be, directs him (her) of action such asstepping a brake.

On the other hand, with the recent advancement of the automatic drivingtechnique, an apparatus has been put to practical used in which a whiteline provided on the road shoulder is detected, and a motor vehicle isautomatically traveled along the white line. Further, there have beenstudied an apparatus in which a distance to a preceding vehicle isdetected by some means, and the motor vehicle is automatically traveledwhile keeping a fixed distance, an apparatus in which the motor vehicleis automatically traveled in accordance with the beacon provided on theroad, and other apparatuses, which will be soon put to practical use(for example, see Japanese Patent Laid-Open No. 162285/1998Publication).

In any way, there have not been present the apparatus for recognizingmarks, signs or the like provided on the road, or the vehicles advancingside by side, passing vehicles, opposing vehicles and the like whichtravel on the roads, and automatically guiding the road along which themotor vehicle travels.

Such a technique as noted above with respect to prior art is alsosimilarly applied to, not only the vehicles traveling on the road suchas a motor vehicle, but also vehicles traveling on the track such astrains, and the body of an airplane navigating the space, or the body ofa ship navigating the sea.

DISCLOSURE OF INVENTION

For automatically recognizing marks, sings or the like provided on theroad, and the circumstances with respect to the own vehicles such asvehicles advancing side by side, passing vehicles, opposing vehicles andthe like which travel on the roads, it is necessary to take in, in alapse of time, the circumstances around the own vehicle as an image, avoice or the like and process the image data, voice data or the liketaken in at high speeds.

However, for automatically recognizing the circumstances around the ownvehicle which change momentarily to automatically guide the road, it isnecessary to process an enormous quantity of image data, voice data andthe like, but it is so difficult to process them at high speeds.Further, it is absolutely impossible to process them at real time.

Such a problem as noted above similarly occurs in, not only the vehiclestraveling on the road such as a motor vehicle, but also vehiclestraveling on the track such as trains, and the body of an airplanenavigating the space, or the body of a ship navigating the sea.

The present invention has been accomplished in view of the conventionalproblem as noted above, and has its object to provide an automatic guideapparatus for traffic facilities capable of processing an enormousquantity of image data, voice data or the like at high speeds,automatically recognizing the momentarily changing circumstances withrespect to traffic facilities, and automatically guiding an operatingroute of the road or the like.

For achieving the above-described object, an automatic guide apparatusfor traffic facilities according to the present invention ischaracterized in comprising: an input device for obtaining thecircumstances around the traffic facilities to be operated as an image,a voice or the like; a database having image data, voice data or thelike in connection with the traffic facilities stored in advance; acomparison device for comparing the image data, the voice data or thelike obtained by said input device with the image data, voice data orthe like stored in said database; a recognition device for recognizingand specifying, where the results obtained by comparing the image data,the voice data or the like coincided, the contents of the data; and anoutput device for informing an operator or the like of the resultsrecognized and specified by the recognition device in a letter ofcharacter, an image, a voice or the like.

Further, preferably, the apparatus comprises a storage device which,where the image data, the voice data or the like corresponding to theobjects obtained by said input device are not present within saiddatabase, makes the image data, the voice data or the like correspondingto new objects correspond to a position on the map to newly store themin said database; and a data update device which, where the image data,the voice data or the like corresponding to the objects are differentfrom the image data, the voice data or the like stored in said database,updates them to new image data, voice data or the like to store them insaid database.

Further, preferably, the apparatus further comprises a judgment devicefor carrying out some judgment on the basis of the matter recognized orspecified by said recognition device to inform an operator or the likeof directions based on the judged results in a letter or character, animage, a voice or the like by said output device.

Further, preferably, said judgment device carries out some judgment onthe basis of the matter recognized or specified by said recognitiondevice, and directs said output device of a fixed action on the basis ofthe judged results to automatically actuate a brake device, a drivingdevice or the like.

Further, in said automatic guide apparatus for traffic facilities, adevice or a plurality of devices constituting it are connected withother devices through communication lines.

Further, there may be included a plane development processing devicecomprising: a plane image conversion device for converting perspectiveimage data with respect to the circumstances around the trafficfacilities obtained by said input device into plane image data having aperspective sense eliminated; a plane image recognition device forrecognizing and specifying, on the basis of results obtained bycomparing the converted plane image data with the image data stored inthe database by said comparison device, the contents of said data; animage content measuring device for various space physical amounts inconnection with the objects recognized and specified by said plane imagerecognition device.

Further, said plane image conversion device may have a function ofconverting image data in the whole periphery of 360 degrees about thecircumstances around the traffic facilities obtained by said inputdevice.

Further, a traffic information detection device for obtaining thecircumstances around the traffic facilities as image data, measuringdata or the like may be installed on the operating route of the trafficfacilities so as to receive the image data, measuring data or the likeobtained by the traffic information detection device.

Preferably, said traffic information detection device may have a graphicdevice for making a computer graphic on the basis of the image data andmeasuring data obtained.

Further, there may be included a position relation recognition devicecomprising an image obtaining portion for obtaining a picture image bythe input device mounted on the traffic facilities, an image temporarilyrecording portion for recording the obtained picture image for a certainperiod, a clue-point automatic extraction portion for automaticallyextracting a clue point for taking a corresponding point within theimage, a corresponding-point detection portion for taking out more thantwo images different in distance to seek for corresponding points of aplurality of clue points in the images, an input-device positionaldirection measuring portion for operating a position and a direction ofhe input device from a plurality of corresponding points detected, andan actual-measurement scale conversion portion for converting a relativedistance value of a three dimensional coordinate of the input deviceposition sought into an absolute distance value using an actualmeasurement value.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a structural view of an automatic guide apparatus for trafficfacilities according to the present invention;

FIG. 2 is an explanatory view showing the operation of the automaticguide apparatus for traffic facilities according to the presentinvention; where the traffic facilities are a motor vehicle;

FIG. 3 is an explanatory view showing the operation of the automaticguide apparatus for traffic facilities according to the presentinvention; where the traffic facilities are a railroad vehicle;

FIG. 4 is an explanatory view showing the operation of the automaticguide apparatus for traffic facilities according to the presentinvention; where the traffic facilities are an airplane;

FIG. 5 is an explanatory view showing the operation of the automaticguide apparatus for traffic facilities according to the presentinvention; where the traffic facilities are a ship;

FIG. 6 is a structural view of a plane development processing apparatusto be mounted on the automatic guide apparatus for traffic facilitiesaccording to the present invention;

FIG. 7 is a view, in a system in which a perspective image obtained by avideo camera is subjected to plane development and converted into aplane image from which a perspective sense is eliminated, showing avariable to be used;

in FIG. 8 and FIG. 9, (A) is a perspective image obtained by an inputdevice, and (B) is a development view converted by a plane imageconversion device;

FIG. 10 is an explanatory view showing a space physical amount to bemeasured by an image content measuring device;

FIG. 11 is an explanatory view showing the concept in which a 360-degreewhole periphery image is subjected to plane development and convertedinto a plane image;

FIG. 12 is an example of a plane image converted from the 360-degreewhole periphery image;

FIG. 13 is an explanatory view showing an installation example of atraffic information detecting device in the road;

FIG. 14 is an explanatory view showing a transmission example from atraffic information detecting device to an automatic guide apparatus fortraffic facilities;

FIG. 15 is an example of an output screen to be displayed as a computergraphic image where a vehicle travel on the road;

FIG. 16 is an example of an output image to be displayed as a graphicimage, where the vehicle moves into a crossing;

FIG. 17 is a structural view of a position relation recognition devicemounted on the automatic guide apparatus for traffic facilitiesaccording to the present invention;

FIG. 18 is a view showing an example of an object portion and partthereof to be selected as a clue point in the position relationrecognition device shown in FIG. 17; and

FIG. 19 is a conceptual view showing a superposed state of a field ofview in a plurality of cameras mounted on a vehicle.

BEST MODE FOR CARRYING OUT THE INVENTION

The preferred embodiments of the automatic guide apparatus for trafficfacilities according to the present invention will be described indetail hereinafter with reference to the drawings.

The automatic guide apparatus for traffic facilities 1 according to thepresent invention comprises, as shown in FIG. 1, an input device 2, adatabase 3, a comparison device 4, a recognition device 5, an outputdevice 6, a judgment device 7, a storage device 8, and a data updatingdevice 9.

The input device 2 is a device for, when the traffic facilities areoperated, obtaining the circumstances around the traffic facilities asan image, a voice or the like. For example, an image and a voice areobtained by a video camera and a microphone, respectively.

In the database 3 are stored in advance, in case of a motor vehicle, aroad mark, a road sign, a traffic guide plate or the like, and imagedata, voice data or the like related to a motor vehicle. In case of atrain, a crossing surface, a crossing signal, a moving obstacle, aplatform or the like, and image data, voice data or the like related toa train are stored. In case of an airplane, an airport shape, a runawayof an airport, an induction path, a structure such as a control tower, amoving obstacle such as a motor vehicle, and image data, voice data orthe like related to an airplane or the like are stored. In case of aship, a harbors shape, a quay of harbors, a wharf, and a fairway mark,and image data, voice data or the like related to a ship or the like arestored.

The comparison device 4 compares image data, voice data or the likeobtained by the input device 2 with image data, voice data or the likestored in the database 3.

The recognition device 5 recognizes and specifies, on the basis of theresults obtained by comparing the image data, the voice data or thelike, contents of the data.

The output device 6 is a device for informing an operator or the like ofthe results recognized and specified by the recognition device 5 in aletter or a character, an image, a voice or the like.

The storage device 8 is a device for allowing, where the image data,voice data or the like corresponding to the objects obtained by theinput device 2 are not present within the database, the image data,voice data or the like corresponding to the new objects correspond to aposition on a map to newly store them in the database 3.

The data updating device 9 is a device for updating, where even if theobject is present on a fixed position on the map, image data, voice dataor the like corresponding to the object are different from the imagedata, voice data or the like stored in the database 3, them to new imagedata, voice data or the like to store them in the database 3.

The judgment device 7 is a device for carrying out, on the basis of thematter recognized or specified by the recognition device 5, somejudgment, wherein the output device 6 informs an operator or the like ofa direction based on the judged results in a letter or character, animage, a voice or the like.

Further, the judgment device 7 sometimes directs the output device 6 ofa fixed action on the basis of judged results to automatically actuate abrake device such as a brake, and a handling device such as a handle.

In the following, a specific description will be made of an example,where the traffic facilities are a motor vehicle, in which the automaticguide apparatus 1 for traffic facilities automatically recognizes thecircumstances of the own vehicle to inform a driver or an occupant ofthe circumstances in a letter or character, an image, a voice or thelike, to guide the road, and to direct adequate action at the time oftravel on the road.

Embodiment 1

As shown in FIG. 2, a road mark 12 whose regulation speed is 50 km isobtained as image data by the input device 2, first, the image data iscompared with the image data stored in the database 3 by the comparisondevice 4, and it is recognized and specified as road mark data whoseregulation speed is 50 km by the recognition device 5.

Then, the fact that the regulation speed is 50 km is informed, on thebasis of the recognized results to a driver or an occupant in a letteror character, an image, a voice or the like by the output device 6.

As described above, according to the automatic guide apparatus fortraffic facilities 1, also where a driver or an occupant overlooks theroad mark 12, it is possible to know positively that in the road beingcurrently traveled, the regulation speed is 50 km to enable preventing aviolation of the traffic regulations caused by the over-speed and alsoto enable preventing an occurrence of a rear-end collision or the like.

Embodiment 2

As shown in FIG. 2, where the other vehicle 13 is obtained as image dataon the right side of the own vehicle 11 by the input device 2, the imagedata is compared with image data stored within the database by thecomparison device 4, and the other vehicle 13 is recognized as a passingvehicle by the recognition device 5.

Then, the judgment device 7 carries out judgment that the other vehicle13 does not accelerate during passing, and the output device 6 informs adriver or an occupant of a direction that not to accelerate in a letteror character, an image, a voice or the like.

As described above, according to the automatic guide apparatus fortraffic facilities 1, also where a driver or an occupant does not becomeaware of the fact that the other vehicle 13 is present on the right sideof the own vehicle 11, the driver or occupant is able to know positivelythat the other vehicle 13 is passing, and it is possible to prevent anoccurrence of trouble such as collision with great presence of mind bythe direction so as not to accelerate.

Embodiment 3

As shown in FIG. 2, where a red lamp present ahead is turned on and offby the input device 2, and a crossing signal 14 at which an alarm isringing is obtained as image data, voice data or the like, first, theyare compared with the image data, voice data or the like stored in thedatabase 3, and the fact that the red lamp of the crossing signal 14ahead is turned on and off and the alarm is ringing is recognized by therecognition device 5.

Then, the judgment device 7 carries out judgment that should be stoppedon the basis of the recognized results, and directs the output device 6to actuate the brake in consideration of the fact that moving into thecrossing is dangerous and prohibited by the law also. Thereby, theoutput device 6 causes the brake to actuate, and therefore, the motorvehicle automatically stops at a safety position before the crossing.

As described above, according to the automatic guide apparatus fortraffic facilities 1, also where a driver or an occupant does not becomeaware of the fact that the red lamp of the crossing signal 14 is turnedon and out and the alarm is ringing ahead, and where even if he becomesaware, he did not take adequate action, the apparatus is able topositively inform that moving into the crossing is dangerous because thered lamp of the crossing signal 14 is turned on and out and the alarm isringing, to direct and execute the adequate action of automaticallystopping, thus enabling prevention of an occurrence of accidence or thelike doing serious damage.

Next, where the traffic facilities are a train, a specific descriptionwill be made of an example wherein the automatic guide apparatus fortraffic facilities 1 automatically recognizes the circumstances of theown vehicle, informs a driver or an occupant of the circumstances in aletter or character, an image, a voice or the like to guide a line or todirect adequate action when traveling on the line.

Embodiment 4

As shown in FIG. 3, where a platform 21 is obtained as image data aheadof the own vehicle by the input device 2, first, it is compared with theimage data stored in the database 3 by the comparison device 4, and thefact that a stop station is near is recognized by the recognition device5.

Then, the judgment device 7 carries out, on the basis of the recognizedresults, judgment that since a stop station is near, deceleration ismade, and the output device 6 informs a driver or an occupant of adirection of making deceleration by an image, a voice or the like.

As described above, according to the automatic guide apparatus fortraffic facilities, also where a driver or an occupant does not becomeaware of the fact that the platform 24 is present ahead of the ownvehicle, it is possible to positively know that a stop station is near,and it is possible to prevent the vehicle from passing the stop stationdue to an error by the direction of making deceleration.

Embodiment 5

As shown in FIG. 3, where a crossing surface 22, a crossing signal 23and a moving obstacle 24 such as a motor vehicle are obtained as imagedata or voice data ahead of the own vehicle by the input device 2,first, they are compared with the image data or the voice data stored inthe database 3, and that fact that a red lamp of a crossing signal 23ahead is turned on and out, and an alarm is ringing, and the fact that amoving obstacle 24 such as a motor vehicle is present at the crossingsurface 22 are recognized by the recognition device 5.

Then, the judgment device 7 carries out, on the basis of the recognizedresults, judgment that the vehicle has to be stopped urgently, anddirects the output device 6 of immediately actuating a brake inconsideration of the fact that moving into the crossing surface 22 isdangerous. Thereby, since the output device 6 immediately actuates abrake, the own vehicle automatically stops urgently at a safety positionbefore the crossing surface 22.

As described above, according to the automatic guide apparatus fortraffic facilities 1, also where a driver or an occupant does not awareof the fact that the red lamp of the crossing signal 23 is turned on andout, and the alarm is ringing, and the fact that the moving obstacle 24such as a motor vehicle is present on the crossing surface 22, andfurther, where even if he becomes aware of it, adequate action has notbeen taken, the apparatus is able to positively inform that the red lampof the crossing signal 23 is turned on and out and the alarm is ringing,and moving into the crossing surface 22 is dangerous, and directadequate action of automatically rapidly stopping the vehicle, thusenabling preventing an occurrence of a crossing accident or the likedoing great damage.

Next, a description will be made specifically of an example whereinwhere the traffic facilities are an airplane, the automatic guideapparatus for traffic facilities 1 automatically recognizes thecircumstances of the own vehicle, informs an operator or an occupant ofthe circumstances in a letter or character, an image, a voice or thelike, guides the navigating route or the interior of an airport, anddirects adequate action when navigating the route and traveling theinterior of the air port.

Embodiment 6

As shown in FIG. 4, where moving obstacles such as the other body 32, amotor vehicle 32 or the like are obtained as image data or voice dataahead of the own body 31 by the input device 2, first, they are comparedwith the image data or the voice data stored within the database 3 bythe comparison device 4, and the fact that the moving obstacles such asthe other body 32, the motor vehicle 32 or the like are present ahead isrecognized by the recognition device 5.

Then, the judgment device 7 carries out, on the basis of the recognizedresults, judgment that the vehicle has to be stopped urgently, anddirects the output device 6 so as to immediately actuate a brake inconsideration of the fact that moving ahead is dangerous. Thereby, sincethe output device 6 causes the brake to be immediately actuated, the ownbody 31 automatically stops urgently at a safety position before themoving obstacle.

As described above, according to the automatic guide apparatus fortraffic facilities 1, also where an operator or an occupant does notbecome aware of the fact that the moving obstacles such as the otherbody 32, the motor vehicle 33 or the like are present ahead of the ownbody 31, and further, where even if becoming aware, adequate action hasnot been taken, the apparatus is able to positively inform that themoving obstacle is present ahead and moving ahead is dangerous, todirect and execute adequate action of automatically stopping urgently,thus enabling preventing an occurrence of accident of an airplane or thelike doing great damage.

Next, a description will be made specifically of an example whereinwhere the traffic facilities are a ship, the automatic guide apparatusfor traffic facilities 1 automatically recognizes the circumstances ofthe own ship, informs an operator or an occupant of the circumstances ina letter or character, an image, a voice or the like, guides thenavigating route or the interior of a harbor, and directs adequateaction when navigating the route and navigating the interior of theharbors.

Embodiment 7

As shown in FIG. 5, where when a ship 41 navigates in order to reach theshore within the harbors 45, a waterway, a wharf 43, and a quay 44 areobtained as image data, first, they are compared with the image data orthe voice data stored within the database 3 by the comparison device 4,and the waterway, the wharf 43, and the quay 44 are recognized by therecognition device 5.

Then, the judgment device 7 judges, on the basis of the recognizedresults, the navigating route from a position reaching the shore of thequay 44 and the current position of the ship 41, and the relative lawsand ordinances such as the harbors law, the maritime traffic safety lawor the like, and the output device 6 informs an operator or an occupantof a direction about the navigating route in a letter or character, animage, a voice or the like.

As described above, according to the automatic guide apparatus fortraffic facilities 1, also where an operator or an occupant isunfamiliar with the shape of the harbors 45, the navigating route to theposition reaching the ashore of the quay 44 can be known easily, thusenabling smoothly reaching the ashore to the quay 44, and enablingrelieving confusion of the ship within the harbors 45.

Embodiment 8

As shown in FIG. 5, where the other ship 42, a wharf 43, a quay 44, ashoal or the like are obtained as image data and voice data ahead of theown ship 41 by the input device 2, first, they are compared with theimage data or the voice data stored within the database 3 by thecomparison device 4, and the fact that the other ship 42, the wharf 43,and the quay 44, shoal or the like are present is recognized by therecognition device 5.

Then, the judgment device 7 judges, on the basis of the recognizedresults, urgent stop or change in route has to be made, and directs theoutput device 6 so as to immediately take action of urgently stopping orchanging the route for the ship 41 in consideration of a danger of beingnavigated as it is. With this, since the output device 6 causes the ship41 to be stopped urgently or to change the route, the ship 41 isautomatically stopped at a safety position or moves to a safetyposition.

As described above, according to the automatic guide apparatus fortraffic facilities 1, also where an operator or an occupant does notbecome aware of the fact that the other ship 42, the wharf 43, and thequay 44, shoal or the like are present ahead, and where even if becomingaware of it, adequate action has not been taken, the apparatus is ableto positively inform that the other ship 42, the wharf 43, and the quay44, shoal or the like are present, and navigation as it is dangerous, todirect and execute adequate action of automatically stopping urgentlyand changing the route, enabling preventing an occurrence of accident ofa ship or the like doing great damage.

Alternatively, in the automatic guide apparatus for traffic facilities1, any one or a plurality of devices out of the input device 2, thedatabase 3, the comparison device 4, the recognition device 5, theoutput device 6, the judgment device 7, the storage device 8 and thedata updating device 9 which constitute the apparatus 1 may connected tothe other devices constituting the automatic guide apparatus for trafficfacilities 1 through communication lines.

According to the structure as described above, it becomes possible toremote-control the traffic facilities in a central control center.

As described above, according to the automatic guide apparatus fortraffic facilities, an enormous quantity of image data, voice data orthe like can be processed at high speeds, and it is possible toautomatically recognize the circumstances around the traffic facilitieswhich momentarily change and to automatically guide the operating route.

Further, in the automatic guide apparatus for traffic facilities 1according to the present invention, there can be added a planedevelopment processing device 51.

The plane development processing device 51 is a device in which aperspective image obtained by the input device 2 about thecircumferences around the operating traffic facilities is subjected toplane development thereby to be converted into a plane image from whichperspective sense is eliminated, on the basis of which measuring processfor a distance, an area or the like is executed, and comprises, as shownin FIG. 6, a plane image conversion device 52, a plane image recognitiondevice 53, a plane image coupling device 54, and an image contentmeasuring device 55.

When the traffic facilities are operated, the circumstances around thetraffic facilities are obtained as a perspective image by the inputdevice 2.

The plane image conversion device 5 is a device for subjecting theobtained perspective image to plane development to be thereby convertedinto a plane image.

The perspective image obtained by a video camera as the input device 2is subjected to plane development by the following Equations (1) and (2)using variables shown in FIG. 7, for example.y=v·2^(1/2) ·h·cos(π/4−θ)·cos(β−θ)/(f·sin β)   (1)x=u·h·cos(β−θ)/(f·sin β)   (2)

Wherein θ is an angle formed between an optical axis of a camera and aplane of the road surface or the like; f is a focal distance of thecamera; h is a height of the camera; β is an angle formed between a linesegment for joining a point in a distance from a portion directly belowthe camera to h+y with the camera, and a plane of the road surface orthe like; v is a coordinate in a longitudinal direction from an originalpoint on the projection surface of the camera; u is a coordinate in alateral direction from an original point on the projection surface ofthe camera; y is a longitudinal coordinate whose original point is apoint moved by h from a portion directly below the camera in the planesuch as the road surface; and x is a lateral coordinate in the planesuch as the road surface.

The plane image recognition device 53 is a device for recognizing andspecifying, on the basis of the results obtained by comparing theconverted plane image data with the image data stored in the database 3,the content of the data.

The plane image coupling device 54 is a device for suitably coupling theconverted plane image data to produce plane image data of a largescreen.

The image convent measuring device 55 is a device for measuring variousspace physical amounts in connection with objects recognized andspecified by the plane image recognition device 53. The space physicalamounts include a position, a distance, an area, a height, a depth, avolume, a speed, an acceleration or the like.

Next, a specific description will be made of an example wherein wherethe traffic facilities are a vehicle traveling on the road such as amotor vehicle, a fixed space physical amount is measured and processedby the plane development processing device 51.

Embodiment 9

As shown in FIG. 8(A), where a traveling lane 56 extending ahead of theown vehicle is obtained as perspective image data by the input device 2,first, the data is converted into plane image data as viewed from thevertical top with a perspective sense eliminated, as shown in FIG. 8(B),by the plane image conversion device 52. Here, the contents of theconverted plane image data are recognized and specified by the planeimage recognition device 53, to recognize and specify the fact that thetraveling lane 56 is present within the plane image.

Then, various space physical amounts of a lane width, a parallelism of alane or the like are measured, in connection with the traveling lane 56as an object, as shown in FIG. 8(B), by the image content measuringdevice 55. Further, if perspective image data is obtained at real timeby the input device 2, a moving distance per time is measured to therebyenable measuring an advancing speed of the own vehicle.

Next, a specific description will be made of an example wherein wherethe traffic facilities are a vehicle traveling on the track such as atrain, a fixed space physical amount is measured and processed by theplane development processing device 51.

Embodiment 10

Where a track 57 extending ahead of the own vehicle is obtained asperspective image data by the input device 2, as shown in FIG. 9(A),first, the data is converted into plane image data as viewed from thevertical top with a perspective sense eliminated, as shown in FIG. 9(B),by the plane image conversion device 52. Here, the contents of theconverted plane image data are recognized and specified by the planeimage recognition device 53, to recognize and specify the fact that thetrack 57 is present within the plane image.

Then, various space physical amounts of a rail width, a rail parallelismor the like are measured, in connection with the track 57 as an object,as shown in FIG. 10, by the image content measuring device 55. Further,if perspective image data is obtained at real time by the input device2, a moving distance per time is measured to thereby enable measuring anadvancing speed of the own vehicle.

As described above, according to the plane development processing device51, which is annexed to the automatic guide apparatus for trafficfacilities 1, various space physical amounts in connection with thecircumstances around the traffic facilities can be obtained whenoperating the traffic facilities.

Further, it is also possible to issue more adequate notice and directionon the basis of the obtained space physical amounts in the automaticguide apparatus for traffic facilities 1.

The plane development processing device 51 may have not only a functionof converting a perspective image into a plane image, but also afunction of converting a 360-degree whole peripheral image (sphericalimage) into a plane image.

The 360-degree whole peripheral image is an image which grasps the wholeperiphery of the operating traffic facilities, that is, all directionsincluding before and behind, left and right, and top and bottom, whichimage can be obtained by methods, for example, {circle around (1)}images photographed by many cameras are synthesized, {circle around (2)}a curved mirror is installed on the front surface of a camera, and animage projected on the curved surface is photographed, {circle around(3)} a single camera is rotated, and images photographed at respectivepositions are synthesized, and {circle around (4)} a fish-eye lens ismounted on the camera, and an image photographed in a wide range isprocessed.

If the 360-degree whole peripheral image is regarded as an imageattached onto an imaginary spherical surface, as shown in FIG. 11, theconverted plane image can be grasped as an image projected on the planeas viewed from a desired visual point.

Where the circumstances around a motor vehicle traveling on the roadsurface are obtained as the 360-degree whole peripheral image by themethods as described above, an example is shown in FIG. 12, in which theimage is converted into a plane image with the top of a motor vehicle asa visual point.

While in the foregoing, a description has been made of examples in whichthe traffic facilities are a vehicle traveling on the road surface suchas a motor vehicle, and a vehicle traveling on the track such as atrain, it is noted that the same is also applied to an airplane, a shipor the like.

Further, alternatively, a traffic information detection device 61 isseparately installed on the operating route of the traffic facilities sothat image data, measured data or the like obtained by the trafficinformation detection device 61 may be transmitted to the automaticguide apparatus for traffic facilities 1.

The traffic information detection device 61 comprises, in the automaticguide apparatus for traffic facilities 1, the input device 2, thedatabase 3, the comparison device 4, the recognition device 5, thestorage device 8, the data updating device 9, the plane developmentprocessing device 51, and a graphical device 62.

The graphical device 62 is a device for forming computer graphic (CC) onthe basis of image data and measured data obtained from the input device2.

Next, a specific description will be made of an example wherein wherethe traffic facilities are a motor vehicle, image data, measured data orthe like obtained by the traffic information detection device 61installed on the operating route are transmitted to the automatic guideapparatus for traffic facilities 1.

Embodiment 11

In case of a motor vehicle, the traffic information detection device 61is installed on a gate 63, an illumination lamp 64 or the like providedon the road as shown in FIG. 13, for example. And, as shown in FIG. 14,image data, measured data or the like obtained by the trafficinformation detection device 61 are, first, transmitted to a basestation 65 installed for the purpose of concentrating and controllingdata, then delivered to a plurality of terminal stations 66 installed onsuitable locations, and transmitted from the terminal stations 66 to therespective vehicles on which the automatic guide apparatus for trafficfacilities 1 is mounted.

The image data, measured data or the like to be delivered from thetraffic information detection device 61 to the respective vehicles onwhich the automatic guide apparatus for traffic facilities 1 is mountedthrough the base station 65 and the terminal stations 66 are formed intocomputer graphic images (CG), which are transmitted together withfigures, numerical values or the like. Therefore, in the automatic guideapparatuses for traffic facilities 1 of the respective vehicles, theyare displayed in figure, numerical value, letter or the like, as shownin FIG. 15, which is very convenient.

In an output screen formed into computer graphic, the coordinate of theown vehicle is fixed to a suitable position, and the circumferencesaround the traveling own vehicle, that is, positions or displays of theroad mark, rod sign, traffic guide plate, other vehicles or the like areto be changed momentarily when passing through the traffic informationdetection device 61.

It is noted that images displayed on the output screens may be, notlimiting to the plane computer graphic image, a three-dimensionalcomputer graphic image (3 DCG) or may be an image synthesized with anactual image subjected to plane development.

An output screen shown in FIG. 15 shows a case where other vehicleswhich infringe or possibly infringe a preset safety zone of the ownvehicle are present, and this informs a driver or an occupant in aletter or character, a voice or the like such as “Passing vehicle ispresent Speed per hour: 140 km”.

Embodiment 12

The recognition of traffic information by transmission from the trafficinformation detection device 61 to the automatic guide apparatus fortraffic facilities 1 makes it possible to easily grasp, particularlywhere in a crossing or the like, left and right roads are the dead pointfrom the own vehicle, as shown in FIG. 16, the traffic circumstances ofthe left and right roads, which is very useful in terms of the trafficsafety measures.

An output screen shown in FIG. 16 shows a case where other vehicles arepresent at a position at which the left and right roads are the deadangle from the own vehicle, and this informs a driver or an occupant ina letter or character, a voice or the like such as “Right-turn vehicleis present ahead”.

While in the foregoing, a description has been made taking an example inwhich the traffic facilities are a motor vehicle traveling on the roadsurface, it is to be noted that this can be likewise applied also to avehicle traveling on the track such as a train, an airplane, a ship orthe like.

Further, if in the automatic guide apparatus for traffic facilities 1, adevice capable of accurately recognizing a positional relation betweenthe operating traffic facilities and the circumference is added, it ispossible automatically guide the traffic facilities more effectively.

In the following, a description will be made of a positional relationrecognition device 101 capable of accurately recognizing a positionalrelation between the operating traffic facilities and the circumference,where traffic facilities are a vehicle traveling on the road surfacesuch as a motor vehicle.

As the apparatus for recognizing a positional relation of an operatingvehicle to guide the vehicle, there has been already developed anapparatus for detecting a while line attached to the road from a pictureobtained by a camera mounted, and guide the vehicle so as not toprotrude from the while line.

In the above apparatus, the own vehicle is positioned only in therelation with the white line, limiting to a special object called awhite line. Therefore, in the road having a white line not attached, aposition recognition function and an automatic guiding function stop,and if a white line is formed to be curved, an erroneous operationpossibly occurs. Further, where advancing along the white line like acrossing is not always in line with the purpose, a position recognitionfunction and an automatic guiding function have no meaning at all.Further, there is a fear that those other than a white line areerroneously recognized as a white line, and even if traveling along thewhite line, where an obstacle is present, it is not possible to avoidit.

The positional relation recognition apparatus 101 according to thepresent invention is an apparatus which recognizes, making use of notonly the white line attached to the road but also all objects in thepicture obtained by a camera, a three dimensional position of the ownvehicle, and which is able to three-dimensionally position the ownvehicle in the peripheral circumstances.

That is, in the positional relation recognition apparatus 101, not onlythe white line attached to the road but also all objects present in thepicture serve as a clue for three dimensional measurement. Further, notthe objects themselves, but a part that is easily extracted by the imageprocess in the objects serves as a clue. Further, a plurality of thosewhich tend to be clue for the three dimensional measurement in theobjects are extracted. Furthermore, the objects are not extracted at theoutset, but there is characterized in that objects that could beextracted serve as a clue.

In the respective images of a continuous image in which the clueportions in the picture are produced by the advancement of the vehicle,an image processing technique such as a matching method is applied forautomatic tracking. The tracking is sometimes succeeded, or the tracingis sometimes failed. However, only the clue portion that succeeded thetracking may be employed as an object of three dimensional measurement.

On the basis of a plurality of clue portions that succeeded the trackingduring the vehicle is moving for suitable distance, a fixed calculationis carried out to thereby detect and recognize the three dimensionalposition and direction of the own vehicle.

Further, a plurality of cameras are mounted so that fields of view arenot superposed, a corresponding point is detected among the superposedfields of view and three dimensional measurement is carried out. And, athree dimensional position and a direction of the own vehicle arepositioned in the three dimensional coordinate constituted by portionsto be peripheral clues, or in the peripheral three dimensional imageobtained by other methods.

According to the positional relation recognition apparatus 101 in thepresent invention,

{circle around (1)} Since a great parallax can be obtained, measuringaccuracy with respect to the distant object is enhanced.

{circle around (2)} Accumulation of errors can be lessened.

{circle around (3)} Since a position of a camera can be measured bycalculation, the structure of apparatus is simple, and setting of acamera is extremely facilitated.

{circle around (4)} Since the three dimensional measurement is enabledfrom picture data photographed by a camera, the past picture materialscan be used.

{circle around (5)} From data with respect to the point to be a featureobtained in the past by traveling, with the feature point as a clue, aposition of the own vehicle can be obtained in a three-dimensionalmanner by calculation.

{circle around (6)} A vehicle position and direction in an area notprojected in the picture can be obtained by calculation.

As described above, according to the positional relation recognitionapparatus 101, various functions that never be done by the positiondetection of the own vehicle merely by the white line can be displayed.

And, the peripheral traffic circumstances are judged, which istransmitted to the operator, or the vehicle is controlled directly onthe basis thereof, and the higher-level traffic guidance can berealized, thus enabling contributing to the traffic safety effectively.

The positional relation recognition apparatus 101 according to thepresent invention is an apparatus which detects, on the basis of pictureinformation of the road surface, the peripheral body or the likephotographed by an output device such as a video camera mounted on thevehicle, a three dimensional position of the road surface, theperipheral body or the like and the own vehicle so as to be able torecognize a positional relation with the road surface. The apparatus ischaracterized in comprising: an image obtaining portion 102 forobtaining an image by an input device mounted on the vehicle; an imagetemporarily recording portion 103 for recording the obtained image for acertain period; a clue point automatic extrusion portion 104 forautomatically extracting a clue point for taking a corresponding pointwithin the image; a corresponding point detection portion 105 fordefecting a corresponding point of a plurality of clue points in therespective images; an input device position and direction measuringportion 106 for calculating a position and direction of the inputdevice; and an actually measured scale conversion portion 107 forconverting a relative distance in a three dimensional coordinate at aposition of the calculated input device into an absolute distance usingthe actually measured value.

According to the mage obtaining portion 102 and the image temporarilyrecording portion 103, a picture from the input device such as a videocamera mounted on the own vehicle, and by arranging them on the timeaxis, an image which changes in time as the vehicle advances isobtained. That is, in connection with a static object, there is obtaineda result similar to that photographed by arranging a plurality ofcameras on the road on which the vehicle advances. By doing so, even asingle camera is used, a plurality of parallaxes can be obtained bymoving the cameras.

Next, by the clue point automatic extracting portion 104, a contour ofan image is extracted in order to make a feature point in an image aclue point, or divided into color areas to automatically extract thefeature portion in the image.

The clue point termed herein means, in order to obtain a correspondingpoint between images photographed at points different n position, afeature point within the image to be the corresponding point. The cluepoint is sometimes an object itself, but is often a portion of theobject. Further, since the feature point is not a feature point for ahuman being but may be a feature point for a computer which processes animage, the image processing is extremely advantageous.

It is not necessary to specify a clue point to be detected at theoutset, but an object that could be detected easily in the actual siteat that time or a portion thereof can be employed as a clue point.Further, with respect to a plurality of clue points detected, it is notnecessary to specify a clue point to be tracked at the outset, but onlythe clue point that could be tracked can be employed as a clue point fora calculation object for a three dimensional measurement. By thesefeatures, the recognition by the positional relation recognitionapparatus 101 can be realized.

Next, by the corresponding point detection portion 105 and the inputdevice position and direction measuring portion 106, a plurality ofcorresponding points are detected by an image matching method or thelike, and an input device position and direction of a camera or the likeis obtained by calculation on the basis of coordinates of the respectivecorresponding points.

Here, the position and direction of the input device are continuouslyobtained, whereby the speed of the input device, that is, the vehiclespeed, the vehicle acceleration, and the vehicle advancing direction canbe obtained.

Some corresponding points are hidden as a vehicle moves, but if manyclue points are sufficiently selected, and several clue points areremained finally, this will suffice to enable calculationsatisfactorily.

Further, an object secured to a static coordinate system has to beselected to a clue point, but sufficiently many clue points areselected, and statistics processing is applied thereto to delete thoseindicative of abnormal values and leave only effective clue points,whereby the three dimensional measurement becomes enabled.

The data of a camera position obtained by calculation is a relativedistance, and for converting it to an absolute distance, one or moreknown distances are necessary in connection with the corresponding pointcoordinate or the vehicle position coordinate which is an object to bemeasured. However, as the value which is not changed in distance as thevehicle travels, for example, a height where a cameral is installed or adistance known at the outset within the image may be used andcalibrated. Further, where a plurality of cameras are installed, thedistance between the cameras can be employed as a known distance.Furthermore, if the known actually measured value is present in theobject within the image, it can be converted to an absolute distance bythe actually measured scale conversion portion 107.

In the way as described above, the camera position and direction can bemeasured in a three-dimensional manner.

To the above-described positional relation recognition device 101 may beadded a corresponding point three dimensional measuring portion 108 forthree dimensional measuring a plurality of clue points fromcorresponding points within the respective images of a plurality of cluepoints to obtain a relation between them and a camera position as athree dimensional coordinate.

According to the corresponding point three dimensional measuring portion108, three dimensional data of a clue point is also obtainedsimultaneously during the calculation for obtaining the threedimensional data of the position and direction of a camera. Thereby, itis possible to arrange a camera position in a three dimensional positionof a plurality of clue points, a three dimensional disposition, and atwo dimensional distribution. That is, the position and direction of avehicle can be arranged in a three dimensional manner in the same threedimensional coordinate system as the three dimensional distribution of aplurality of clue points in the periphery.

Whereby, a position of the own vehicle, correctly, a camera mounted onthe own vehicle can be positioned in a three dimensional manner in astatic object in a static coordinate system in the periphery, in otherwords, in a three dimensional coordinate including peripheral buildings,poles, roads or the like.

Further, there can be added a three dimensional data recording portion109 for recording a three dimensional coordinate of a correspondingpoint obtained by the corresponding point three dimensional measuringportion 108.

The three dimensional data of the clue point obtained by the calculationis recorded and left, whereby when traveling around later, the cluepoint having obtained three dimensional coordinate already may bereferred to as an index of the calculation of the position and directionof the vehicle.

Actually, a vehicle having an exclusive-use device mounted in advance istraveled, and three dimensional data of more accurate clue points arecollected, whereby many vehicles may use them later.

Further, there can be added a three dimensional data read-out portion110 for reading out three dimensional data of clue points accumulated inthe three dimensional data recording portion 109 from the threedimensional data recording portion 109 when traveling in the peripherythereafter; and a corresponding point comparison portion 111 forcomparing those data with the three dimensional data obtained whentraveling thereafter to obtain a coincided point, thus enhancingcalculation accuracy of a vehicle position.

The three dimensional data are read out by the three dimensional dataread-out portion 110 from the three dimensional data recording portion109, and if there is a change in a coordinate of a previous clue pointevery time of later travel, its coordinate may be updated and recordedagain as new data.

As the travel near the same place is repeated, the data of the threedimensional data recording portion 109 are updated and added newly,whereby the clue points are increased, or the position accuracy isenhanced, and the calculation accuracy of the position and direction ofthe vehicle is also enhanced.

The three dimensional data of high accuracy are produced by theexclusive-use device in advance, which are used later by other vehicles,whereby the position accuracy of the clue point is enhanced extremely.Further, if three dimensional map data having a clue point expanded to apixel unit are produced in advance by the exclusive-use device, a threedimensional space around the travel path is constituted, within whichposition and direction of a vehicle can be arranged.

Further, there can be added an absolute coordinate conversion portion112 for selecting an object whose absolute coordinate is known at acorresponding point, and imparting an absolute coordinate to threedimensional data obtained by the input device direction measuringportion 106 and the corresponding point measuring portion 108; and acoordinate unifying portion 113 for unifying a three dimensionalcoordinate of a clue point present in an area relative to an absolutecoordinate system.

Acceding to the absolute coordinate conversion portion 112, for example,an absolute coordinate of a camera position is obtained by GPS, or threedimensional data obtained as a relative distance can be converted to anabsolute coordinate with an object whose latitude and longitude areknown as a clue.

Further, according to the coordinate unifying portion 113, coordinatesof some clue points can be unified in a common absolute coordinatesystem and displayed, and the three dimensional data is to obtain theabsolute coordinate immediately.

If the clue point obtains the absolute coordinate, thereafter also, oreven other vehicles can use it as three dimensional data whose cluepoint is common.

Site data coincided with three dimensional data having clue pointscollected is to have obtained an absolute coordinate because of beingcoincided with a position, disposition and distribution of clue points.Then, a camera position, that is, an absolute coordinate of a vehiclecan be obtained immediately.

Further, a three-dimensional coordinate by way of a clue point isdropped into a map of the whole area whereby a map and a clue point areconnected to create new data, and many vehicles may obtain a map forrecording data of position and direction of the own vehicle as commondata.

Further, there can be added a name attribute adding portion 114 forcorresponding a name and attribute of a clue point to position data of aclue point to record and store them, and adding a name and attribute ofan object to which a clue point belongs to coordinate data of the cluepoint; and a database 115 for relating a coordinate, name and attributeof the added clue point into a map to write, record and store them.

If the name and attribute of the object to which a clue point belongsare known in advance, or if a name is known by the image recognition,the name of the object, its general properties, or properties peculiarto the object can be also obtained as attribute data.

That is, if the clue point at the time of measurement is corresponded tothe clue point in the database, the name and properties of the object atthe clue point can be read.

Further, there can be also added a display portion 116 for suitablydisplaying and informing a driver or the like of various calculationresults as described above.

If the obtained data, position and direction of the vehicle, the nameand attribute of the object to which clue point belongs or the like aredisplayed by the display portion 116, the driver or a manager observesthem to enable judging the circumstances.

There can be added a circumstance judgment portion 117 for automaticallyjudging the circumstances of a vehicle from a positional relationbetween the periphery and the own vehicle and a positional relationbetween the road and the own vehicle by the calculation results; and anautomatic vehicle control portion 118 for automatically carrying outoperations (such as brake operation, speed control operation, handleoperation, alarm operation or the like) suited to an object of a vehicleautomatically, using the results obtained by judging the circumstances.

According to the circumstance judgment portion 117, the position anddirection of the own vehicle, the name and attribute of the object towhich clue point belongs, name and attribute of the clue point,positional relation between the road surface and the vehicle, mark, roadsign or the like are synthetically judged to enable judging thecircumstances of the vehicle.

Whereby, the vehicle can be operated automatically or semi-automaticallythrough the automatic vehicle control portion 118. Further, thepositional relation between the circumstances of the periphery and thevehicle is judged to enable transmitting position information of thevehicle to a driver or the like.

Further, there can be added a plural camera image obtaining portion 119for installing a plurality of cameras to take in images, and superposingpart or the whole field of view of the respective cameras; and acalibration portion 120 for jointly using both three dimensionaldistance measurement calculated from the parallax caused by a movingdistance of a single camera and three dimensional distance measurementcalculated from the parallax between cameras by a plurality of cameras,calibrating the three dimensional distance data obtained by the movingdistance parallax system by movement of a single camera, with threedimensional distance data of clue points obtained by the field of viewsuperposing system of a plurality of cameras as a reference length, tothereby convert the former to an absolute distance.

If a plurality of cameras are installed on the vehicle so as to surroundthe vehicle so that the circumference of the vehicle may be observed,the field of view of each camera is partly superposed to that of othercameras, it is possible to generate the parallax between cameras in thesuperposed portion.

The greatest characteristics of the three dimensional distancemeasurement by the plural camera field of view superposing system isthat even the moving body can be measured. Further, according to theplural camera field of view superposing system, high accuracy is createdin the measurement of a short distance.

On the other hand, according to the single camera moving distancesystem, since the distance between cameras in the plural camera systemcan be made long distance substantially, it is advantageous when a threedimensional distance of long distance is measured. For example, adistance between cameras in the camera mounted on the vehicle is about 1m at most, but a distance between cameras in the camera moving distancesystem can be made not only 1 m, 10 m, 100 m but also 1 km, 10 km.

So, the short distance is subjected to three dimensional measurement bythe field of view superposing system, and the long distance is subjectedto three dimensional measurement by the camera moving distance system bywhich the parallax is taken greatly.

Fundamentally, the principle of measurement is the same in case ofsingle and in case of plural. However, as the characteristics of theplural camera system in which fields of view are superposed, thedistance between cameras can be taken as a reference length in themeasurement by the parallax, because of which absolute distancemeasurement can be done; moreover, since the measured distance by thesingle camera moving system can be calibrated by distance data ofintermediate distance measurement obtained from the distance measurementby the parallax, accurate measurement from short distance to longdistance becomes enabled. That is, a position, speed, acceleration, andadvancing direction of a vehicle traveling ahead can be measured.

Further, there can be added a shape, coordinate, and attribute displayportion 120 for reproducing a three dimensional space, which arrangesand expresses a three dimensional shape of an object to which a cluepoint belongs at a correct position in a coordinate system defined in adisplay screen by a three dimensional computer graphics, and displays aname, attribute, other objects, and the own vehicle in the displayscreen as necessary; and a user interface portion 121 for designating anintended object by touching it by a hand or clicking by a mouse on theimage of the three dimensional computer graphics showing various objectsexpressed in the display screen, or by touching by a hand or clicking bya mouse the intended object on the actual image displaying only theactual image, whereby enabling displaying a name, coordinate, shape, andattribute of other related data of the object, inputting data related tothe designated intended object, and directing the designated intendedobject various operations or actions.

Where a plurality of clue points were able to obtain a three dimensionalcoordinate, name and attribute of the object to which the clue pointsbelong, these clue points mean that the clue points correspond to therespective objects to which they belong.

So, the three dimensional shape and three dimensional coordinate of theobject are put into the database in advance whereby the own vehicle andthe circumstances of the circumference thereof can be expressed by 3 DCG(three dimensional computer graphics) and displayed in the threedimensional coordinate system. Of course, even objects whose attributeand clue point are not found can be displayed since its shape andposition coordinate are known.

The judgment of circumstances can be carried out more adequately fromthe reconstructed three dimensional space and information of a positionrelation, name, attribute or the like of the objects arranged therein.

Further, by clicking by a mouse or touching by a hand the objectdisplayed on the 3 DCG screen to designate it, or directing it by avoice directly and understanding the directed content by the voicerecognition device, the attribute of the intended object can be read outfrom the database and displayed.

Furthermore, two dimensional data of the actual image and projectedimage data to the two dimensional of the three dimensional computergraphics are put one other the other so that the shape is coincided inan air line to display only the actual image, and the object image onthe display screen is clicked by a mouse or touched by a hand todesignate the intended object. Then, data of the corresponding object in3 DCG is designated from the data construction which is constituted suchthat the actual image and the 3 DCG image are coincided so as to besuperposed to each other to enable calling or writing the name,coordinate, attribute, and other related information in connection withthe object.

Further, there can be added an external communication portion 122connected to other vehicles or other communication points through acommunication line to transmit or receive information.

If it is connected to other vehicles or other communication pointsthrough a communication line, it is possible that a direction is madeautomatically, or through a user interface, so that three dimensionalinformation of a clue point produced by the own vehicle, and moving bodyinformation are transmitted to others, or three dimensional informationand moving body information of a clue point produced by other vehiclesand at a different position, similarly to that the own vehicle produces,are received from other vehicles.

Further, it is possible that a fixed station transmits clue pointinformation analyzed by a picture obtained by a camera installed at aperipheral fixed point, position information of a vehicle including theown vehicle, speed information and information of the result obtained byjudging the circumstances, for example, traffic jam information,accident information or the like, the own vehicle receives and displaysthese information, or receives and displays even information that cannotbe obtained by the own vehicle, and received information is added toinformation obtained by the own vehicle to judge the circumstances, todisplay information of high accuracy.

Next, a specific description will be made of an example, where trafficfacilities are a vehicle traveling on the road surface such as a motorvehicle, in which a positional relation between the operating trafficfacilities and the periphery is recognized by the positional relationrecognition device 101.

Embodiment 13

As shown in FIG. 19, four super wide angle cameras having a fish-eyelens on a roof portion of a vehicle are installed so that fields of vieware partly superposed, and distance measurement and speed measurement ofthe vehicle were carried out. Further, with respect to the clue point ofthe short distance, three dimensional measurement was carried out by theparallax between a plurality of cameras; and with respect to the cluepoint of the long distance, a corresponding point of the clue pointcaused by movement of a single camera was detected to carry out threedimensional measurement on the basis of the moving distance.

The calculation by the parallax between cameras as well as thecalculation by the moving parallax caused by movement of a camerabasically comprise the calculation of the same type, but are differentwhich unknown quantity is. In the case of the moving parallax, themoving distance of a camera is an unknown quantity; and in the case bythe parallax between cameras, since the distance between cameras can bemeasured actually in advance, it can be an unknown quantity.

Further, if three dimensional measurement data obtained by thecalculation is recorded, when traveling in the same area next, the cluepoint obtained previously is an unknown quantity, to further enhance theaccuracy, thus enabling three dimensional measurement. An example of theclue point is shown by a symbol ◯ in FIG. 18.

The development of the obtained picture into the spherical coordinatetype attached to the spherical surface is advantageous for a processingto be done later. For the positional relation of cameras, it is desiredthat distance between cameras and direction be measured accurately, butthe distance between cameras can be also obtained as an unknown quantityby calculation, and therefore, it may be installed suitably. Thedistance between cameras can be also used as a reference of actualmeasurement value conversion.

The direction of a camera can be also an unknown quantity, but where itis expected that the direction is somewhat deviated due to vibrations orthe like depending on the fixing method of a camera, the direction of acamera can be handled as an unknown quantity.

Where there are many unknown quantities, may clue points are prepared,and these unknown quantities can be obtained by calculation. Inprinciple, the clue points can be increased to the number of pixels.

One or more clue points are accurately measured by the. parallax ofcameras whose fields of view are superposed to use it as a referencevalue, which can be converted into an actual measurement scale, or aheight from the road surface of a camera is used as a reference value,which can be converted into an actual measurement scale. Further, anobject in which a length in an image is known can be also made as areference length.

Since the parallax is obtained in a portion where fields of view aresuperposed, a corresponding point of a clue point can be obtained byimage recognition in a picture of cameras photographed simultaneouslywith a distance between cameras as a reference length. The clue pointcan be automatically extracted at real time from the image by contourprocessing of an image. A cross intersection, a triangular intersection,a square intersection or the like are extracted from an image formedinto a contour in order to let image recognition have a real timeproperty and in order to facilitate calculation. For collecting data ofclue points, the real time property is not necessary, and considerablymany clue points can be obtained by calculation through on-lineprocessing. Further, it is possible to expand to the whole surface ofpixel.

In general, some out of many clue points are made clue point having afeature point, and a corresponding point is obtained by an imagematching method or the like. If the corresponding point is decided, athree dimensional distance to the clue point can be obtained bycalculation. At the same time, a camera position and direction areobtained by calculation.

All the clue points are not always found as a corresponding point.Supposing a case where a corresponding point cannot be found halfway,preferably, somewhat many clue points are automatically extracted.

Preferably, where a clue point was found, a three dimensional positionof the clue point is corresponded to a map, by post processing, forexample, and name and attribute of an object to which a clue pointbelongs are linked and recorded in a database.

Thereby, in travels thereafter, a clue point is merely found, and a nameof a corresponding point to which the clue point belongs and anattribute owned by the object can be known.

If the database is utilized, a position of a clue point is merelyobtained to thereby make it possible to recognize an object to which aclue point belongs, understand the circumstances of a vehicle, and judgean action that should be taken next. And, the vehicle's direction andspeed are suitably controlled by the judged results to enableautomatically guiding the vehicle properly.

Further, if the result of three dimensional measurement, or the resultof judgment are output in an image display or a voice, it is possible toinform a driver or a manager as shown in FIG. 15. Further, ifinformation from other vehicles and information obtained from a fixedcamera installed in the periphery are synthesized, the circumstances inthe periphery of the own vehicle can be displayed, and it is possible toinform a driver or a manager. In any case, the display screen functionsas a user interface to enable designating an intended vehicle, startingcommunication, transmitting and receiving data, and clearly indicatingthe vehicle.

While in the foregoing, a description has been made taking, as anexample, a vehicle traveling on the road surface such as a motor vehicleas traffic facilities, it is noted that also in a vehicle traveling onthe track such as a train, an airplane, a ship, a spaceship or the like,the above matter can be likewise applied thereto.

1. An automatic guide apparatus for traffic facilities, comprising: aninput device for obtaining the circumstances around the trafficfacilities to be operated as an image, a voice or the like; a databasehaving image data, voice data or the like in connection with the trafficfacilities stored in advance; a comparison device for comparing theimage data, the voice data or the like obtained by said input devicewith the image data, voice data or the like stored in said database; arecognition device for recognizing and specifying, where the resultsobtained by comparing the image data, the voice data or the likecoincided, the contents of the data; and an output device for informingan operator or the like of the results recognized and specified by therecognition device in a letter of character, an image, a voice or thelike.
 2. The automatic guide apparatus for traffic facilities accordingto claim 1, further comprising: a storage device which, where the imagedata, the voice data or the like corresponding to the objects obtainedby said input device are not present within said database, makes theimage data, the voice data or the like corresponding to new objectscorrespond to a position on the map to newly store them in saiddatabase; and a data update device which, where the image data, thevoice data or the like corresponding to the objects are different fromthe image data, the voice data or the like stored in said database,updates them to new image data, voice data or the like to store them insaid database.
 3. The automatic guide apparatus for traffic facilitiesaccording to claim 1 or 2, further comprising a judgment device forcarrying out some judgment on the basis of the matter recognized orspecified by said recognition device to inform an operator or the likeof directions based on the judged results in a letter or character, animage, a voice or the like by said output device.
 4. The automatic guideapparatus for traffic facilities according to claim 3, wherein saidjudgment device carries out some judgment on the basis of the matterrecognized or specified by said recognition device, and directs saidoutput device of a fixed action on the basis of the judged results toautomatically actuate a brake device, a driving device or the like. 5.The automatic guide apparatus for traffic facilities according to claims1 to 4, wherein one or a plurality of devices are connected with otherdevices through communication lines.
 6. An automatic guide apparatus fortraffic facilities, traffic facilities to be operated being vehiclestraveling on the road surface such as motor vehicle, comprising: aninput device for obtaining the circumstances around the own vehicle asan image, a voice or the like; a database having image data, voice dataor the like in connection with the motor vehicle or the like such asroad marks, road signs, traffic guide plates or the like stored inadvance; a comparison device for comparing the image data, the voicedata or the like obtained by said input device with the image data,voice data or the like stored in said database; a recognition device forrecognizing and specifying, where the results obtained by comparing theimage data, the voice data or the like coincided, the contents of thedata; and an output device for informing a driver or an occupant of theresults recognized and specified by the recognition device in a letterof character, an image, a voice or the like.
 7. An automatic guideapparatus for traffic facilities, traffic facilities to be operatedbeing vehicles traveling on the track of a railroad train of the like,comprising: an input device for obtaining the circumstances around theown vehicle as an image, a voice or the like; a database having imagedata, voice data or the like in connection with the railroad vehicles orthe like such as track marks, track signs, track guide plates or thelike stored in advance; a comparison device for comparing the imagedata, the voice data or the like obtained by said input device with theimage data, voice data or the like stored in said database; arecognition device for recognizing and specifying, where the resultsobtained by comparing the image data, the voice data or the likecoincided, the contents of the data; and an output device for informinga driver or an occupant of the results recognized and specified by therecognition device in a letter of character, an image, a voice or thelike.
 8. An automatic guide apparatus for traffic facilities, trafficfacilities to be operated being the ship body or vehicle body navigatingthe two dimensional or three dimensional without the track of a ship, anairplane or the like, comprising: an input device for obtaining thecircumstances around the own vehicle as an image, a voice or the like; adatabase having image data, voice data or the like in connection withthe ship or airplane or the like such as marks, signs, the shape ofharbors, the shape of an airport or the like stored in advance; acomparison device for comparing the image data, the voice data or thelike obtained by said input device with the image data, voice data orthe like stored in said database; a recognition device for recognizingand specifying, where the results obtained by comparing the image data,the voice data or the like coincided, the contents of the data; and anoutput device for informing a driver or an occupant of the resultsrecognized and specified by the recognition device in a letter ofcharacter, an image, a voice or the like.
 9. The automatic guideapparatus for traffic facilities according to claims 1 to 8, furthercomprising: a plane development processing device comprising a planeimage conversion device for converting perspective image data withrespect to the circumstances around the traffic facilities obtained bysaid input device into plane image data having a perspective senseeliminated; a plane image recognition device for recognizing andspecifying, on the basis of results obtained by comparing the convertedplane image data with the image data stored in the database by saidcomparison device, the contents of said data; an image content measuringdevice for various space physical amounts in connection with the objectsrecognized and specified by said plane image recognition device.
 10. Theautomatic guide apparatus for traffic facilities according to claim 9,further comprising: a plane development processing device comprising animage content measuring device for measuring various space physicalamounts in connection with the objects recognized and specified by saidplane image recognition device.
 11. The automatic guide apparatus fortraffic facilities according to claim 9 or 10, wherein said plane imageconversion device has a function of converting image data in the wholeperiphery of 360 degrees about the circumstances around the trafficfacilities obtained by said input device.
 12. The automatic guideapparatus for traffic facilities according to claims 1 to 11, wherein atraffic information detection device for obtaining the circumstancesaround the traffic facilities as image data, measuring data or the likeis installed on the operating route of the traffic facilities so as toenable receiving the image data, measuring data or the like obtained bythe traffic information detection device.
 13. The automatic guideapparatus for traffic facilities according to claim 12, wherein saidtraffic information detection device ha a graphic device for making acomputer graphic on the basis of the image data and measuring dataobtained.
 14. The automatic guide apparatus for traffic facilitiesaccording to claims 1 to 13, further comprising: a position relationrecognition device comprising an image obtaining portion for obtaining apicture image by the input device mounted on the traffic facilities, animage temporarily recording portion for recording the obtained pictureimage for a certain period, a clue-point automatic extraction portionfor automatically extracting a clue point for taking a correspondingpoint within the image, a corresponding-point detection portion fortaking out more than two images different in distance to seek forcorresponding points of a plurality of clue points in the images, aninput-device positional direction measuring portion for operating aposition and a direction of he input device from a plurality ofcorresponding points detected, and an actual-measurement scaleconversion portion for converting a relative distance value of a threedimensional coordinate of the input device position sought into anabsolute distance value using an actual measurement value.
 15. Theautomatic guide apparatus for traffic facilities according to claim 14,wherein to said position relation recognition device is added acorresponding point three dimensional measuring portion for 3-dimensionmeasuring a plurality of clue points from corresponding points in theimages of a plurality of clue points to obtain a relation between themand a position of the input device as a three dimensional coordinate.16. The automatic guide apparatus for traffic facilities according toclaim 15, wherein to said position relation recognition device is addeda three dimensional data recording portion for recording a threedimensional coordinate of a corresponding point obtained by saidcorresponding point three dimensional measuring portion.
 17. Theautomatic guide apparatus for traffic facilities according to claim 16,wherein to said position relation recognition device are added a threedimensional data read-out portion for reading out three dimensional dataof a clue point accumulated in said three dimensional data recordingportion obtained by being operated from the three dimensional datarecording portion at the time of operating the peripheries after thesucceeding time, and a corresponding point comparison portion forcomparing those data with image data obtained at the time of operationsafter the succeeding time to obtain a coincident point to therebyenhance operation accuracy of a position of the traffic facilities. 18.The automatic guide apparatus for traffic facilities according to claim17, wherein to said position relation recognition device are added anabsolute coordinate conversion portion for selecting an object whoseabsolute coordinate is known to a corresponding point to impart anabsolute coordinate to three dimensional data obtained in said inputdevice position direction measuring portion and said corresponding pointthree dimensional measuring portion; and a coordinate synthesizingportion for synthesizing a three dimensional coordinate of a clue pointpresent in a certain area to an absolute coordinate system.
 19. Theautomatic guide apparatus for traffic facilities according to claim 18,wherein to said position relation recognition device are added a nameattribute adding portion for corresponding a name and an attribute of aclue point to position data of a clue point to record and store them,and adding names and attributes of the objects to which the clue pointsbelong to the coordinate data of said clue points; and a database forrelating a coordinate, a name and a an attribute of the added clue pointinto a map to write, record and store them.
 20. The automatic guideapparatus for traffic facilities according to claim 19, wherein to saidposition relation recognition device is added a display portion forsuitably displaying and informing an operator or the like of saidvarious operation results.
 21. The automatic guide apparatus for trafficfacilities according to claim 19, wherein to said position relationrecognition device are added a circumstances judgment portion forautomatically judging the circumferences of the traffic facilities fromthe periphery and the position relation of the traffic facilities to beoperated by said various operation results; and an automatic controlportion for automatically carrying out an operation automatically suitedto an object of the traffic facilities using the results of obtained byjudging the circumstances.
 22. The automatic guide apparatus for trafficfacilities according to claim 21, wherein to said position relationrecognition device are added a plural-input device image obtainingportion for, a plurality of input devices being installed, taking inimages, and superposing a part or the whole of a field of view of therespective input devices; and a calibration portion for, using boththree dimensional distance measurement calculated from a difference inview caused by an input movement distance by a single input device andthree dimensional distance measurement calculated from a distance inview between the input devices by a plurality of input devices, withthree dimensional distance data of clue points obtained by a viewsuperposing system of a plurality of input devices as a referencelength, calibrating three dimensional distance data obtained by amovement distance difference-in-view by the single input device movementto thereby convert it into an absolute distance.
 23. The automatic guideapparatus for traffic facilities according to claim 22, wherein to saidposition relation recognition device are added a shape attribute displayportion for reproducing a three dimensional space for arranging andexpressing a three dimensional shape of an object to which a clue pointbelongs in a three dimensional computer graphic at a proper position ina coordinate system defined in a display screen to enable displayingeven a name, an attribute, other objects, and the own vehicle in theirdisplay images; and a user interface portion for touching by a hand orclicking by a mouse on the image of the three dimensional computergraphic showing various objects expressed in said display image, ordisplaying only the actual image, and touching by a hand or clicking bya mouse the intended object on the displayed actual picture to designatethe intended object, thereby inputting data related to the intendedobject which displays and directs a name, a coordinate, a shape, andattributes of other related data of the object to enable directing thedirected intended object various operations and actions.
 24. Theautomatic guide apparatus for traffic facilities according to claim 23,wherein to said position relation recognition device is added anexternal communication portion which is connected to other trafficfacilities or other communication points through communication lines toreceive and deliver information.